In a speech-enabled environment, such as a home or automobile, a user may access information and/or control various functions using voice input. The information and/or functions may be personalized for a given user. In multiple user environments, it may therefore be advantageous to identify a given speaker from among a group of speakers associated with the speech-enabled environment.